Face Generation

In this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate new images of faces that look as realistic as possible!

The project will be broken down into a series of tasks from loading in data to defining and training adversarial networks. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.

Get the Data

You'll be using the CelebFaces Attributes Dataset (CelebA) to train your adversarial networks.

This dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.

Pre-processed Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.

If you are working locally, you can download this data by clicking here

This is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data processed_celeba_small/

In [1]:
# can comment out after executing
#!unzip processed_celeba_small.zip
In [2]:
data_dir = 'processed_celeba_small/'

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np
import problem_unittests as tests
import helper

%matplotlib inline

Visualize the CelebA Data

The CelebA dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with 3 color channels (RGB)#RGB_Images) each.

Pre-process and Load the Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This pre-processed dataset is a smaller subset of the very large CelebA data.

There are a few other steps that you'll need to transform this data and create a DataLoader.

Exercise: Complete the following get_dataloader function, such that it satisfies these requirements:

  • Your images should be square, Tensor images of size image_size x image_size in the x and y dimension.
  • Your function should return a DataLoader that shuffles and batches these Tensor images.

ImageFolder

To create a dataset given a directory of images, it's recommended that you use PyTorch's ImageFolder wrapper, with a root directory processed_celeba_small/ and data transformation passed in.

In [3]:
# necessary imports
import torch
from torchvision import datasets
from torchvision import transforms
from torchvision import datasets
In [4]:
def get_dataloader(batch_size, image_size, data_dir='processed_celeba_small/'):
    """
    Batch the neural network data using DataLoader
    :param batch_size: The size of each batch; the number of images in a batch
    :param img_size: The square size of the image data (x, y)
    :param data_dir: Directory where image data is located
    :return: DataLoader with batched data
    """
    
    # TODO: Implement function and return a dataloader
    # Tensor transform
    transform = transforms.Compose([
        transforms.Resize(image_size),
        transforms.ToTensor()    
    ])    
    
    dataset = datasets.ImageFolder(root=data_dir, transform=transform)
    
    loader = torch.utils.data.DataLoader(dataset=dataset,
                                          batch_size=batch_size,
                                          shuffle=True)
    
    return loader

Create a DataLoader

Exercise: Create a DataLoader celeba_train_loader with appropriate hyperparameters.

Call the above function and create a dataloader to view images.

  • You can decide on any reasonable batch_size parameter
  • Your image_size must be 64. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!
In [5]:
# Define function hyperparameters
batch_size = 128
img_size = 64

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# Call your function and get a dataloader
celeba_train_loader = get_dataloader(batch_size, img_size)

Next, you can view some images! You should seen square images of somewhat-centered faces.

Note: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested imshow code is below, but it may not be perfect.

In [6]:
import matplotlib.pyplot as plt
import numpy as np
# helper display function
def imshow(img):
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# obtain one batch of training images
dataiter = iter(celeba_train_loader)
images, _ = dataiter.next() # _ for no labels

# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(20, 4))
plot_size=20
for idx in np.arange(plot_size):
    ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])

Exercise: Pre-process your image data and scale it to a pixel range of -1 to 1

You need to do a bit of pre-processing; you know that the output of a tanh activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)

In [7]:
# TODO: Complete the scale function
def scale(x, feature_range=(-1, 1)):
    ''' Scale takes in an image x and returns that image, scaled
       with a feature_range of pixel values from -1 to 1. 
       This function assumes that the input x is already scaled from 0-1.'''
    # assume x is scaled to (0, 1)
    # scale to feature_range and return scaled x
    x = feature_range[0] + x * (  feature_range[1] - feature_range[0] )
    return x
In [8]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# check scaled range
# should be close to -1 to 1
img = images[0]
scaled_img = scale(img)

print('Min: ', scaled_img.min())
print('Max: ', scaled_img.max())
Min:  tensor(-1.)
Max:  tensor(1.)

Define the Model

A GAN is comprised of two adversarial networks, a discriminator and a generator.

Discriminator

Your first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with normalization. You are also allowed to create any helper functions that may be useful.

Exercise: Complete the Discriminator class

  • The inputs to the discriminator are 64x64x3 tensor images
  • The output should be a single value that will indicate whether a given image is real or fake
In [45]:
import torch.nn as nn
import torch.nn.functional as F
In [10]:
class Discriminator(nn.Module):       

    def __init__(self, conv_dim):
        """
        Initialize the Discriminator Module
        :param conv_dim: The depth of the first convolutional layer
        """
        super(Discriminator, self).__init__()

        # complete init function        
        self.conv_dim = conv_dim
        
        from helper import conv

        # From 3 x 64 x 64 to conv_dim x 32 x 32
        self.conv1 = conv(in_channels=3, out_channels=conv_dim, kernel_size=4, batch_norm=False)
        # From conv_dim x 32 x 32 to (conv_dim*2) x 16 x 16
        self.conv2 = conv(conv_dim, conv_dim*2, 4, batch_norm=True)        
        # From (conv_dim*2) x 16 x 16 to (conv_dim*4) x 8 x 8
        self.conv3 = conv(conv_dim*2, conv_dim*4, 4, batch_norm=True)
        # From (conv_dim*4) x 8 x 8 to (conv_dim*8) x 4 x 4
        self.conv4 = conv(conv_dim*4, conv_dim*8, 4, batch_norm=True)
        # From (conv_dim*8) x 4 x 4 -> 1
        self.fc = nn.Linear(conv_dim*8 * (4 * 4), 1)
        

    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: Discriminator logits; the output of the neural network
        """
        # define feedforward behavior
        neg_slope = 0.2
        x = F.leaky_relu(self.conv1(x), neg_slope)
        x = F.leaky_relu(self.conv2(x), neg_slope)
        x = F.leaky_relu(self.conv3(x), neg_slope)
        x = F.leaky_relu(self.conv4(x), neg_slope)
        x = x.view(-1, self.conv_dim*8 *(4*4) )
        
        x = self.fc(x)
        
        return x


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
#tests.test_discriminator(Discriminator)
Out[10]:
"\nDON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n"

Generator

The generator should upsample an input and generate a new image of the same size as our training data 64x64x3. This should be mostly transpose convolutional layers with normalization applied to the outputs.

Exercise: Complete the Generator class

  • The inputs to the generator are vectors of some length z_size
  • The output should be a image of shape 64x64x3
In [11]:
class Generator(nn.Module):
    
    def __init__(self, z_size, conv_dim):
        """
        Initialize the Generator Module
        :param z_size: The length of the input latent vector, z
        :param conv_dim: The depth of the inputs to the *last* transpose convolutional layer
        """
        super(Generator, self).__init__()

        # complete init function
        self.z_size = z_size
        self.conv_dim = conv_dim
        
        self.fc = nn.Linear(z_size, conv_dim*8 *(4*4) )
        
        from helper import deconv
        # From (conv_dim*8) x 4 x 4 to (conv_dim*4) x 8 x 8
        self.t_conv1 = deconv(in_channels=conv_dim*8, out_channels=conv_dim*4, kernel_size=4, batch_norm=True)
        # From (conv_dim*4) x 8 x 8 to (conv_dim*2) x 16 x 16
        self.t_conv2 = deconv(conv_dim*4, conv_dim*2, 4, batch_norm=True)
        # From (conv_dim*2) x 16 x 16 to conv_dim x 32 x 32
        self.t_conv3 = deconv(conv_dim*2, conv_dim, 4, batch_norm=True)
        # From conv_dim x 64 x 64 to 3 x 64 x 64
        self.t_conv4 = deconv(conv_dim, 3, 4, batch_norm=False )
        
    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: A 32x32x3 Tensor image as output
        """
        # define feedforward behavior
        x = self.fc(x)
        x = x.view(-1, self.conv_dim*8, 4, 4)
        x = F.relu(self.t_conv1(x))
        x = F.relu(self.t_conv2(x))
        x = F.relu(self.t_conv3(x))
        x = torch.tanh(self.t_conv4(x))
        
        return x

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
#tests.test_generator(Generator)
Out[11]:
"\nDON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n"

Initialize the weights of your networks

To help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the original DCGAN paper, they say:

All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.

So, your next task will be to define a weight initialization function that does just this!

You can refer back to the lesson on weight initialization or even consult existing model code, such as that from the networks.py file in CycleGAN Github repository to help you complete this function.

Exercise: Complete the weight initialization function

  • This should initialize only convolutional and linear layers
  • Initialize the weights to a normal distribution, centered around 0, with a standard deviation of 0.02.
  • The bias terms, if they exist, may be left alone or set to 0.
In [12]:
def weights_init_normal(m):
    """
    Applies initial weights to certain layers in a model .
    The weights are taken from a normal distribution 
    with mean = 0, std dev = 0.02.
    :param m: A module or layer in a network    
    """
    # classname will be something like:
    # `Conv`, `BatchNorm2d`, `Linear`, etc.
    classname = m.__class__.__name__
    
    # TODO: Apply initial weights to convolutional and linear layers
    if ( (classname.find('Linear') != -1) or 
        (classname.find('Conv2d') != -1) or 
        (classname.find('ConvTranspose2d') != -1)
        ):
        # get the number of the inputs
        y = 0.02
        m.weight.data.normal_(0, y)
        if (m.bias is not None):
            m.bias.data.fill_(0)        

Build complete network

Define your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.

In [13]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
def build_network(d_conv_dim, g_conv_dim, z_size):
    # define discriminator and generator
    D = Discriminator(d_conv_dim)
    G = Generator(z_size=z_size, conv_dim=g_conv_dim)

    # initialize model weights
    D.apply(weights_init_normal)
    G.apply(weights_init_normal)

    print(D)
    print()
    print(G)
    
    return D, G

Exercise: Define model hyperparameters

In [14]:
# Define model hyperparams
d_conv_dim = 64
g_conv_dim = 64
z_size = 100

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
D, G = build_network(d_conv_dim, g_conv_dim, z_size)
Discriminator(
  (conv1): Sequential(
    (0): Conv2d(3, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
  )
  (conv2): Sequential(
    (0): Conv2d(64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (conv3): Sequential(
    (0): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (conv4): Sequential(
    (0): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (fc): Linear(in_features=8192, out_features=1, bias=True)
)

Generator(
  (fc): Linear(in_features=100, out_features=8192, bias=True)
  (t_conv1): Sequential(
    (0): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (t_conv2): Sequential(
    (0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (t_conv3): Sequential(
    (0): ConvTranspose2d(128, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (t_conv4): Sequential(
    (0): ConvTranspose2d(64, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
  )
)

Training on GPU

Check if you can train on GPU. Here, we'll set this as a boolean variable train_on_gpu. Later, you'll be responsible for making sure that

  • Models,
  • Model inputs, and
  • Loss function arguments

Are moved to GPU, where appropriate.

In [15]:
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch

# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
    print('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Training on GPU!')
    print(torch.cuda.get_device_name(0))
Training on GPU!
GeForce GTX 950M

Discriminator and Generator Losses

Now we need to calculate the losses for both types of adversarial networks.

Discriminator Losses

  • For the discriminator, the total loss is the sum of the losses for real and fake images, d_loss = d_real_loss + d_fake_loss.
  • Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.

Generator Loss

The generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to think its generated images are real.

Exercise: Complete real and fake loss functions

You may choose to use either cross entropy or a least squares error loss to complete the following real_loss and fake_loss functions.

In [16]:
def real_loss(D_out, smooth=False):
    batch_size = D_out.size(0)
    # label smoothing
    if smooth:
        # smooth, real labels = 0.9
        labels = torch.ones(batch_size)*0.9
    else:
        labels = torch.ones(batch_size) # real labels = 1
    # move labels to GPU if available     
    if train_on_gpu:
        labels = labels.cuda()
    # binary cross entropy with logits loss
    criterion = nn.BCEWithLogitsLoss()
    # calculate loss
    loss = criterion(D_out.squeeze(), labels)
    return loss

def fake_loss(D_out):
    batch_size = D_out.size(0)
    labels = torch.zeros(batch_size) # fake labels = 0
    if train_on_gpu:
        labels = labels.cuda()
    criterion = nn.BCEWithLogitsLoss()
    # calculate loss
    loss = criterion(D_out.squeeze(), labels)
    return loss

Optimizers

Exercise: Define optimizers for your Discriminator (D) and Generator (G)

Define optimizers for your models with appropriate hyperparameters.

In [17]:
import torch.optim as optim

lr = 0.0002
beta1=0.5
beta2=0.999

# Create optimizers for the discriminator and generator
d_optimizer = optim.Adam(D.parameters(), lr, [beta1, beta2])
g_optimizer = optim.Adam(G.parameters(), lr, [beta1, beta2])

Training

Training will involve alternating between training the discriminator and the generator. You'll use your functions real_loss and fake_loss to help you calculate the discriminator losses.

  • You should train the discriminator by alternating on real and fake images
  • Then the generator, which tries to trick the discriminator and should have an opposing loss function

Saving Samples

You've been given some code to print out some loss statistics and save some generated "fake" samples.

Exercise: Complete the training function

Keep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.

In [19]:
def train(D, G, n_epochs, print_every=50):
    '''Trains adversarial networks for some number of epochs
       param, D: the discriminator network
       param, G: the generator network
       param, n_epochs: number of epochs to train for
       param, print_every: when to print and record the models' losses
       return: D and G losses'''
    
    # move models to GPU
    if train_on_gpu:
        D.cuda()
        G.cuda()

    # keep track of loss and generated, "fake" samples
    samples = []
    losses = []

    # Get some fixed data for sampling. These are images that are held
    # constant throughout training, and allow us to inspect the model's performance
    sample_size=16
    fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))
    fixed_z = torch.from_numpy(fixed_z).float()
    # move z to GPU if available
    if train_on_gpu:
        fixed_z = fixed_z.cuda()

    # epoch training loop
    for epoch in range(n_epochs):

        # batch training loop
        for batch_i, (real_images, _) in enumerate(celeba_train_loader):

            batch_size = real_images.size(0)
            real_images = scale(real_images)

            # ===============================================
            #         YOUR CODE HERE: TRAIN THE NETWORKS
            # ===============================================
            
            d_optimizer.zero_grad()
            
            # 1. Train the discriminator on real and fake images
            
            # 1.a train on real images
            if (train_on_gpu):
                real_images = real_images.cuda()
            
            D_real = D(real_images)
            d_real_loss = real_loss(D_real)
            
            # 2.b train on fake images
            z = np.random.uniform(-1, 1, size=(batch_size, z_size) )
            z = torch.from_numpy(z).float()
            
            if (train_on_gpu):
                z = z.cuda()
                
            fake_images = G(z)
            D_fake = D(fake_images)
            d_fake_loss = fake_loss(D_fake)
                        
            d_loss = d_real_loss + d_fake_loss
            d_loss.backward()
            d_optimizer.step()            

            # 2. Train the generator with an adversarial loss
                                    
            g_optimizer.zero_grad()
            
            z = np.random.uniform(-1, 1, size=(batch_size, z_size) )
            z = torch.from_numpy(z).float()

            if (train_on_gpu):
                z = z.cuda()
            
            # First Optimization of Generator
            fake_images = G(z)
            D_fake = D(fake_images)
            g_loss = real_loss(D_fake) # use real loss to flip labels

            # perform backprop
            g_loss.backward(retain_graph=True)
            g_optimizer.step()                

            # Second Optimization of Generator
            D_fake = D(fake_images)
            g_loss = real_loss(D_fake) # use real loss to flip labels

            # perform backprop
            g_loss.backward()
            g_optimizer.step()                

            # ===============================================
            #              END OF YOUR CODE
            # ===============================================

            # Print some loss stats
            if batch_i % print_every == 0:
                # append discriminator loss and generator loss
                losses.append((d_loss.item(), g_loss.item()))
                # print discriminator and generator loss
                print('Epoch [{:5d}/{:5d}] ({}%) | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(
                        epoch+1, n_epochs, round(100 * batch_i / len(celeba_train_loader)), d_loss.item(), g_loss.item()))


        ## AFTER EACH EPOCH##    
        # this code assumes your generator is named G, feel free to change the name
        # generate and save sample, fake images
        G.eval() # for generating samples
        samples_z = G(fixed_z)
        samples.append(samples_z)        
        G.train() # back to training mode
        
        # Save training generator samples
        with open('train_samples_'+str(epoch)+'.pkl', 'wb') as f:
            pkl.dump(samples, f)

    # Save training generator samples
    #with open('train_samples.pkl', 'wb') as f:
    #    pkl.dump(samples, f)
    
    # finally return losses
    return losses

Set your number of training epochs and train your GAN!

In [20]:
# set number of epochs 
n_epochs = 20

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# call training function
losses = train(D, G, n_epochs=n_epochs)
Epoch [    1/   20] (0%) | d_loss: 1.5784 | g_loss: 1.1763
Epoch [    1/   20] (7%) | d_loss: 1.0021 | g_loss: 2.7619
Epoch [    1/   20] (14%) | d_loss: 1.1811 | g_loss: 3.2161
Epoch [    1/   20] (21%) | d_loss: 1.2893 | g_loss: 1.4900
Epoch [    1/   20] (28%) | d_loss: 1.1153 | g_loss: 1.9896
Epoch [    1/   20] (36%) | d_loss: 1.2718 | g_loss: 2.3400
Epoch [    1/   20] (43%) | d_loss: 1.2605 | g_loss: 1.8262
Epoch [    1/   20] (50%) | d_loss: 1.1474 | g_loss: 2.0183
Epoch [    1/   20] (57%) | d_loss: 1.3327 | g_loss: 1.2877
Epoch [    1/   20] (64%) | d_loss: 1.2296 | g_loss: 2.3794
Epoch [    1/   20] (71%) | d_loss: 1.2643 | g_loss: 2.1306
Epoch [    1/   20] (78%) | d_loss: 1.2637 | g_loss: 2.0131
Epoch [    1/   20] (85%) | d_loss: 1.0558 | g_loss: 1.4338
Epoch [    1/   20] (92%) | d_loss: 1.1682 | g_loss: 1.0424
Epoch [    1/   20] (100%) | d_loss: 1.1938 | g_loss: 1.9191
Epoch [    2/   20] (0%) | d_loss: 1.2788 | g_loss: 2.4275
Epoch [    2/   20] (7%) | d_loss: 1.1882 | g_loss: 1.0994
Epoch [    2/   20] (14%) | d_loss: 1.5472 | g_loss: 2.3630
Epoch [    2/   20] (21%) | d_loss: 1.5164 | g_loss: 2.5054
Epoch [    2/   20] (28%) | d_loss: 1.4486 | g_loss: 1.1160
Epoch [    2/   20] (36%) | d_loss: 1.2852 | g_loss: 1.1485
Epoch [    2/   20] (43%) | d_loss: 1.4660 | g_loss: 1.8324
Epoch [    2/   20] (50%) | d_loss: 1.0404 | g_loss: 1.2525
Epoch [    2/   20] (57%) | d_loss: 1.0819 | g_loss: 1.5584
Epoch [    2/   20] (64%) | d_loss: 1.2392 | g_loss: 0.9412
Epoch [    2/   20] (71%) | d_loss: 1.1167 | g_loss: 1.1884
Epoch [    2/   20] (78%) | d_loss: 1.1318 | g_loss: 1.1517
Epoch [    2/   20] (85%) | d_loss: 1.0467 | g_loss: 1.1024
Epoch [    2/   20] (92%) | d_loss: 1.2476 | g_loss: 1.2885
Epoch [    2/   20] (100%) | d_loss: 1.1325 | g_loss: 1.6491
Epoch [    3/   20] (0%) | d_loss: 1.1867 | g_loss: 1.0918
Epoch [    3/   20] (7%) | d_loss: 1.0704 | g_loss: 1.3044
Epoch [    3/   20] (14%) | d_loss: 1.1401 | g_loss: 1.3664
Epoch [    3/   20] (21%) | d_loss: 1.0290 | g_loss: 1.2579
Epoch [    3/   20] (28%) | d_loss: 1.0591 | g_loss: 1.2899
Epoch [    3/   20] (36%) | d_loss: 1.3199 | g_loss: 1.5334
Epoch [    3/   20] (43%) | d_loss: 1.2522 | g_loss: 1.3878
Epoch [    3/   20] (50%) | d_loss: 1.0696 | g_loss: 1.2424
Epoch [    3/   20] (57%) | d_loss: 1.3089 | g_loss: 0.5440
Epoch [    3/   20] (64%) | d_loss: 1.0716 | g_loss: 0.8501
Epoch [    3/   20] (71%) | d_loss: 1.1496 | g_loss: 1.2229
Epoch [    3/   20] (78%) | d_loss: 1.2161 | g_loss: 1.3231
Epoch [    3/   20] (85%) | d_loss: 1.2032 | g_loss: 1.3031
Epoch [    3/   20] (92%) | d_loss: 1.1455 | g_loss: 1.0193
Epoch [    3/   20] (100%) | d_loss: 1.1807 | g_loss: 0.9518
Epoch [    4/   20] (0%) | d_loss: 1.1576 | g_loss: 1.4489
Epoch [    4/   20] (7%) | d_loss: 1.0782 | g_loss: 1.4571
Epoch [    4/   20] (14%) | d_loss: 1.5109 | g_loss: 1.9701
Epoch [    4/   20] (21%) | d_loss: 1.1588 | g_loss: 1.7225
Epoch [    4/   20] (28%) | d_loss: 1.1713 | g_loss: 1.8286
Epoch [    4/   20] (36%) | d_loss: 1.0729 | g_loss: 0.8406
Epoch [    4/   20] (43%) | d_loss: 1.1980 | g_loss: 0.8176
Epoch [    4/   20] (50%) | d_loss: 1.0638 | g_loss: 1.6828
Epoch [    4/   20] (57%) | d_loss: 1.1901 | g_loss: 1.1994
Epoch [    4/   20] (64%) | d_loss: 1.0662 | g_loss: 1.2564
Epoch [    4/   20] (71%) | d_loss: 1.0772 | g_loss: 1.7113
Epoch [    4/   20] (78%) | d_loss: 1.1070 | g_loss: 1.0011
Epoch [    4/   20] (85%) | d_loss: 1.1226 | g_loss: 1.3899
Epoch [    4/   20] (92%) | d_loss: 1.5994 | g_loss: 2.5544
Epoch [    4/   20] (100%) | d_loss: 1.0316 | g_loss: 1.2412
Epoch [    5/   20] (0%) | d_loss: 1.0692 | g_loss: 1.3801
Epoch [    5/   20] (7%) | d_loss: 1.2751 | g_loss: 1.0896
Epoch [    5/   20] (14%) | d_loss: 1.0251 | g_loss: 1.2128
Epoch [    5/   20] (21%) | d_loss: 1.1059 | g_loss: 1.6191
Epoch [    5/   20] (28%) | d_loss: 1.0724 | g_loss: 1.7506
Epoch [    5/   20] (36%) | d_loss: 0.9163 | g_loss: 1.2932
Epoch [    5/   20] (43%) | d_loss: 1.1181 | g_loss: 0.9810
Epoch [    5/   20] (50%) | d_loss: 1.0554 | g_loss: 1.5791
Epoch [    5/   20] (57%) | d_loss: 1.1478 | g_loss: 0.8059
Epoch [    5/   20] (64%) | d_loss: 1.0728 | g_loss: 1.0739
Epoch [    5/   20] (71%) | d_loss: 1.0046 | g_loss: 1.8069
Epoch [    5/   20] (78%) | d_loss: 1.1175 | g_loss: 2.0979
Epoch [    5/   20] (85%) | d_loss: 0.9967 | g_loss: 1.1176
Epoch [    5/   20] (92%) | d_loss: 1.0865 | g_loss: 1.6128
Epoch [    5/   20] (100%) | d_loss: 0.9794 | g_loss: 1.3781
Epoch [    6/   20] (0%) | d_loss: 0.9026 | g_loss: 1.4196
Epoch [    6/   20] (7%) | d_loss: 0.8521 | g_loss: 1.4800
Epoch [    6/   20] (14%) | d_loss: 0.9824 | g_loss: 1.3584
Epoch [    6/   20] (21%) | d_loss: 1.0511 | g_loss: 1.1396
Epoch [    6/   20] (28%) | d_loss: 1.1905 | g_loss: 0.8285
Epoch [    6/   20] (36%) | d_loss: 1.1466 | g_loss: 0.8023
Epoch [    6/   20] (43%) | d_loss: 1.3598 | g_loss: 0.5833
Epoch [    6/   20] (50%) | d_loss: 1.7595 | g_loss: 3.3476
Epoch [    6/   20] (57%) | d_loss: 0.9406 | g_loss: 2.0601
Epoch [    6/   20] (64%) | d_loss: 0.7913 | g_loss: 1.7882
Epoch [    6/   20] (71%) | d_loss: 0.9290 | g_loss: 1.0867
Epoch [    6/   20] (78%) | d_loss: 1.0074 | g_loss: 1.0604
Epoch [    6/   20] (85%) | d_loss: 1.0882 | g_loss: 2.3128
Epoch [    6/   20] (92%) | d_loss: 1.0095 | g_loss: 2.2907
Epoch [    6/   20] (100%) | d_loss: 1.0205 | g_loss: 3.0456
Epoch [    7/   20] (0%) | d_loss: 0.9749 | g_loss: 1.3587
Epoch [    7/   20] (7%) | d_loss: 0.8664 | g_loss: 1.3204
Epoch [    7/   20] (14%) | d_loss: 0.8623 | g_loss: 1.6662
Epoch [    7/   20] (21%) | d_loss: 0.9089 | g_loss: 1.8902
Epoch [    7/   20] (28%) | d_loss: 0.7442 | g_loss: 1.5275
Epoch [    7/   20] (36%) | d_loss: 0.8811 | g_loss: 2.4375
Epoch [    7/   20] (43%) | d_loss: 0.8210 | g_loss: 1.5066
Epoch [    7/   20] (50%) | d_loss: 0.7555 | g_loss: 1.5814
Epoch [    7/   20] (57%) | d_loss: 1.0229 | g_loss: 1.0686
Epoch [    7/   20] (64%) | d_loss: 0.7317 | g_loss: 2.3059
Epoch [    7/   20] (71%) | d_loss: 0.8373 | g_loss: 2.1100
Epoch [    7/   20] (78%) | d_loss: 0.8554 | g_loss: 2.1777
Epoch [    7/   20] (85%) | d_loss: 0.7612 | g_loss: 1.8775
Epoch [    7/   20] (92%) | d_loss: 0.9223 | g_loss: 3.0212
Epoch [    7/   20] (100%) | d_loss: 0.8561 | g_loss: 1.9619
Epoch [    8/   20] (0%) | d_loss: 0.9504 | g_loss: 3.0137
Epoch [    8/   20] (7%) | d_loss: 0.7077 | g_loss: 1.5133
Epoch [    8/   20] (14%) | d_loss: 0.8847 | g_loss: 3.5546
Epoch [    8/   20] (21%) | d_loss: 0.7546 | g_loss: 1.9529
Epoch [    8/   20] (28%) | d_loss: 0.6958 | g_loss: 2.4402
Epoch [    8/   20] (36%) | d_loss: 0.6463 | g_loss: 2.4029
Epoch [    8/   20] (43%) | d_loss: 0.7516 | g_loss: 2.4060
Epoch [    8/   20] (50%) | d_loss: 0.7993 | g_loss: 1.5910
Epoch [    8/   20] (57%) | d_loss: 0.5231 | g_loss: 2.1513
Epoch [    8/   20] (64%) | d_loss: 0.5665 | g_loss: 2.1146
Epoch [    8/   20] (71%) | d_loss: 0.9985 | g_loss: 5.0597
Epoch [    8/   20] (78%) | d_loss: 1.0895 | g_loss: 2.9746
Epoch [    8/   20] (85%) | d_loss: 0.7051 | g_loss: 3.0193
Epoch [    8/   20] (92%) | d_loss: 0.6531 | g_loss: 1.3904
Epoch [    8/   20] (100%) | d_loss: 0.6414 | g_loss: 1.9166
Epoch [    9/   20] (0%) | d_loss: 0.9256 | g_loss: 0.9646
Epoch [    9/   20] (7%) | d_loss: 0.8352 | g_loss: 2.7663
Epoch [    9/   20] (14%) | d_loss: 0.6781 | g_loss: 1.9381
Epoch [    9/   20] (21%) | d_loss: 0.8065 | g_loss: 3.0271
Epoch [    9/   20] (28%) | d_loss: 0.8277 | g_loss: 3.8919
Epoch [    9/   20] (36%) | d_loss: 0.4476 | g_loss: 1.7297
Epoch [    9/   20] (43%) | d_loss: 0.5862 | g_loss: 1.9695
Epoch [    9/   20] (50%) | d_loss: 1.3838 | g_loss: 0.8597
Epoch [    9/   20] (57%) | d_loss: 0.5511 | g_loss: 2.6566
Epoch [    9/   20] (64%) | d_loss: 0.4785 | g_loss: 1.7132
Epoch [    9/   20] (71%) | d_loss: 1.0581 | g_loss: 0.8908
Epoch [    9/   20] (78%) | d_loss: 0.7761 | g_loss: 1.2181
Epoch [    9/   20] (85%) | d_loss: 0.3496 | g_loss: 2.6645
Epoch [    9/   20] (92%) | d_loss: 0.4783 | g_loss: 3.0767
Epoch [    9/   20] (100%) | d_loss: 0.5261 | g_loss: 2.6235
Epoch [   10/   20] (0%) | d_loss: 0.4103 | g_loss: 2.1045
Epoch [   10/   20] (7%) | d_loss: 0.2973 | g_loss: 3.2017
Epoch [   10/   20] (14%) | d_loss: 0.7363 | g_loss: 2.0355
Epoch [   10/   20] (21%) | d_loss: 0.4666 | g_loss: 2.5394
Epoch [   10/   20] (28%) | d_loss: 0.4355 | g_loss: 3.1282
Epoch [   10/   20] (36%) | d_loss: 0.3298 | g_loss: 2.7020
Epoch [   10/   20] (43%) | d_loss: 0.4820 | g_loss: 2.8019
Epoch [   10/   20] (50%) | d_loss: 1.3607 | g_loss: 5.1972
Epoch [   10/   20] (57%) | d_loss: 0.3477 | g_loss: 2.3193
Epoch [   10/   20] (64%) | d_loss: 1.0649 | g_loss: 7.2257
Epoch [   10/   20] (71%) | d_loss: 0.5278 | g_loss: 2.4198
Epoch [   10/   20] (78%) | d_loss: 0.3787 | g_loss: 2.2864
Epoch [   10/   20] (85%) | d_loss: 0.8601 | g_loss: 1.6377
Epoch [   10/   20] (92%) | d_loss: 0.8318 | g_loss: 1.4014
Epoch [   10/   20] (100%) | d_loss: 0.3609 | g_loss: 3.5791
Epoch [   11/   20] (0%) | d_loss: 0.2822 | g_loss: 3.4836
Epoch [   11/   20] (7%) | d_loss: 0.4454 | g_loss: 2.1150
Epoch [   11/   20] (14%) | d_loss: 0.3373 | g_loss: 3.6970
Epoch [   11/   20] (21%) | d_loss: 0.3174 | g_loss: 3.2087
Epoch [   11/   20] (28%) | d_loss: 0.7176 | g_loss: 5.7272
Epoch [   11/   20] (36%) | d_loss: 0.1664 | g_loss: 3.3774
Epoch [   11/   20] (43%) | d_loss: 0.5566 | g_loss: 2.2780
Epoch [   11/   20] (50%) | d_loss: 0.5170 | g_loss: 1.8384
Epoch [   11/   20] (57%) | d_loss: 0.2212 | g_loss: 2.9206
Epoch [   11/   20] (64%) | d_loss: 2.1228 | g_loss: 0.2434
Epoch [   11/   20] (71%) | d_loss: 0.5764 | g_loss: 2.7515
Epoch [   11/   20] (78%) | d_loss: 0.3208 | g_loss: 2.2809
Epoch [   11/   20] (85%) | d_loss: 0.2572 | g_loss: 2.2995
Epoch [   11/   20] (92%) | d_loss: 0.7334 | g_loss: 2.8536
Epoch [   11/   20] (100%) | d_loss: 0.3836 | g_loss: 3.7425
Epoch [   12/   20] (0%) | d_loss: 0.5073 | g_loss: 1.0339
Epoch [   12/   20] (7%) | d_loss: 0.2807 | g_loss: 2.8098
Epoch [   12/   20] (14%) | d_loss: 0.1652 | g_loss: 2.4506
Epoch [   12/   20] (21%) | d_loss: 0.5208 | g_loss: 3.4945
Epoch [   12/   20] (28%) | d_loss: 0.2707 | g_loss: 2.5223
Epoch [   12/   20] (36%) | d_loss: 0.1897 | g_loss: 3.0861
Epoch [   12/   20] (43%) | d_loss: 0.3534 | g_loss: 2.3642
Epoch [   12/   20] (50%) | d_loss: 0.2642 | g_loss: 2.2645
Epoch [   12/   20] (57%) | d_loss: 1.8441 | g_loss: 1.9100
Epoch [   12/   20] (64%) | d_loss: 0.2936 | g_loss: 2.5996
Epoch [   12/   20] (71%) | d_loss: 0.2983 | g_loss: 1.9033
Epoch [   12/   20] (78%) | d_loss: 0.4347 | g_loss: 2.7807
Epoch [   12/   20] (85%) | d_loss: 0.6056 | g_loss: 1.5454
Epoch [   12/   20] (92%) | d_loss: 0.4021 | g_loss: 3.7206
Epoch [   12/   20] (100%) | d_loss: 0.1862 | g_loss: 3.4673
Epoch [   13/   20] (0%) | d_loss: 0.1590 | g_loss: 3.1178
Epoch [   13/   20] (7%) | d_loss: 0.2738 | g_loss: 3.7606
Epoch [   13/   20] (14%) | d_loss: 0.0720 | g_loss: 3.3386
Epoch [   13/   20] (21%) | d_loss: 0.5394 | g_loss: 1.2311
Epoch [   13/   20] (28%) | d_loss: 0.3960 | g_loss: 1.6711
Epoch [   13/   20] (36%) | d_loss: 0.1051 | g_loss: 3.6499
Epoch [   13/   20] (43%) | d_loss: 1.0835 | g_loss: 1.0751
Epoch [   13/   20] (50%) | d_loss: 1.6052 | g_loss: 2.0427
Epoch [   13/   20] (57%) | d_loss: 1.0126 | g_loss: 1.0481
Epoch [   13/   20] (64%) | d_loss: 0.3928 | g_loss: 2.1206
Epoch [   13/   20] (71%) | d_loss: 0.2381 | g_loss: 3.1482
Epoch [   13/   20] (78%) | d_loss: 0.3781 | g_loss: 4.3122
Epoch [   13/   20] (85%) | d_loss: 0.1248 | g_loss: 2.6149
Epoch [   13/   20] (92%) | d_loss: 0.1834 | g_loss: 3.4217
Epoch [   13/   20] (100%) | d_loss: 0.5068 | g_loss: 1.6598
Epoch [   14/   20] (0%) | d_loss: 0.4094 | g_loss: 2.2671
Epoch [   14/   20] (7%) | d_loss: 0.1192 | g_loss: 3.7152
Epoch [   14/   20] (14%) | d_loss: 0.1469 | g_loss: 2.5649
Epoch [   14/   20] (21%) | d_loss: 0.9446 | g_loss: 1.4064
Epoch [   14/   20] (28%) | d_loss: 1.7497 | g_loss: 1.9403
Epoch [   14/   20] (36%) | d_loss: 0.4386 | g_loss: 4.4029
Epoch [   14/   20] (43%) | d_loss: 1.1143 | g_loss: 2.2258
Epoch [   14/   20] (50%) | d_loss: 1.1726 | g_loss: 0.7416
Epoch [   14/   20] (57%) | d_loss: 0.3439 | g_loss: 3.1928
Epoch [   14/   20] (64%) | d_loss: 0.6794 | g_loss: 2.4983
Epoch [   14/   20] (71%) | d_loss: 0.1468 | g_loss: 2.9542
Epoch [   14/   20] (78%) | d_loss: 0.1329 | g_loss: 4.0300
Epoch [   14/   20] (85%) | d_loss: 0.1692 | g_loss: 4.1879
Epoch [   14/   20] (92%) | d_loss: 0.0968 | g_loss: 3.4244
Epoch [   14/   20] (100%) | d_loss: 0.1659 | g_loss: 3.1306
Epoch [   15/   20] (0%) | d_loss: 0.1080 | g_loss: 3.5765
Epoch [   15/   20] (7%) | d_loss: 0.1052 | g_loss: 3.7683
Epoch [   15/   20] (14%) | d_loss: 0.5560 | g_loss: 2.7117
Epoch [   15/   20] (21%) | d_loss: 0.5236 | g_loss: 2.4551
Epoch [   15/   20] (28%) | d_loss: 0.2158 | g_loss: 3.0338
Epoch [   15/   20] (36%) | d_loss: 0.1771 | g_loss: 1.9979
Epoch [   15/   20] (43%) | d_loss: 0.7492 | g_loss: 1.3280
Epoch [   15/   20] (50%) | d_loss: 0.1524 | g_loss: 3.3991
Epoch [   15/   20] (57%) | d_loss: 0.0922 | g_loss: 4.7168
Epoch [   15/   20] (64%) | d_loss: 0.0790 | g_loss: 3.1491
Epoch [   15/   20] (71%) | d_loss: 0.1311 | g_loss: 4.2428
Epoch [   15/   20] (78%) | d_loss: 0.1566 | g_loss: 3.5800
Epoch [   15/   20] (85%) | d_loss: 1.3378 | g_loss: 5.6204
Epoch [   15/   20] (92%) | d_loss: 0.2679 | g_loss: 2.4164
Epoch [   15/   20] (100%) | d_loss: 0.2692 | g_loss: 1.9906
Epoch [   16/   20] (0%) | d_loss: 0.6723 | g_loss: 5.2756
Epoch [   16/   20] (7%) | d_loss: 0.0615 | g_loss: 4.5506
Epoch [   16/   20] (14%) | d_loss: 0.6719 | g_loss: 2.5292
Epoch [   16/   20] (21%) | d_loss: 0.0887 | g_loss: 3.3458
Epoch [   16/   20] (28%) | d_loss: 0.2166 | g_loss: 2.5046
Epoch [   16/   20] (36%) | d_loss: 1.8717 | g_loss: 1.7331
Epoch [   16/   20] (43%) | d_loss: 0.1227 | g_loss: 3.6297
Epoch [   16/   20] (50%) | d_loss: 0.1167 | g_loss: 3.7218
Epoch [   16/   20] (57%) | d_loss: 0.0694 | g_loss: 4.0370
Epoch [   16/   20] (64%) | d_loss: 0.0535 | g_loss: 4.9760
Epoch [   16/   20] (71%) | d_loss: 0.0651 | g_loss: 4.4250
Epoch [   16/   20] (78%) | d_loss: 0.2650 | g_loss: 3.6986
Epoch [   16/   20] (85%) | d_loss: 0.2026 | g_loss: 2.9410
Epoch [   16/   20] (92%) | d_loss: 0.1276 | g_loss: 3.4364
Epoch [   16/   20] (100%) | d_loss: 0.1785 | g_loss: 4.7723
Epoch [   17/   20] (0%) | d_loss: 0.1473 | g_loss: 4.5470
Epoch [   17/   20] (7%) | d_loss: 0.4801 | g_loss: 1.1885
Epoch [   17/   20] (14%) | d_loss: 0.1262 | g_loss: 4.2101
Epoch [   17/   20] (21%) | d_loss: 0.3135 | g_loss: 6.0390
Epoch [   17/   20] (28%) | d_loss: 0.1184 | g_loss: 3.2962
Epoch [   17/   20] (36%) | d_loss: 0.1839 | g_loss: 4.3315
Epoch [   17/   20] (43%) | d_loss: 0.1095 | g_loss: 5.0046
Epoch [   17/   20] (50%) | d_loss: 1.0041 | g_loss: 1.2267
Epoch [   17/   20] (57%) | d_loss: 0.1890 | g_loss: 4.6136
Epoch [   17/   20] (64%) | d_loss: 0.2576 | g_loss: 4.2217
Epoch [   17/   20] (71%) | d_loss: 0.5122 | g_loss: 1.9703
Epoch [   17/   20] (78%) | d_loss: 0.2858 | g_loss: 2.1215
Epoch [   17/   20] (85%) | d_loss: 0.1120 | g_loss: 3.6656
Epoch [   17/   20] (92%) | d_loss: 1.5623 | g_loss: 5.2565
Epoch [   17/   20] (100%) | d_loss: 0.0798 | g_loss: 3.6236
Epoch [   18/   20] (0%) | d_loss: 0.0608 | g_loss: 3.9339
Epoch [   18/   20] (7%) | d_loss: 0.0857 | g_loss: 5.8779
Epoch [   18/   20] (14%) | d_loss: 0.3014 | g_loss: 4.0700
Epoch [   18/   20] (21%) | d_loss: 0.1175 | g_loss: 5.0940
Epoch [   18/   20] (28%) | d_loss: 0.3240 | g_loss: 3.2878
Epoch [   18/   20] (36%) | d_loss: 0.4831 | g_loss: 6.5080
Epoch [   18/   20] (43%) | d_loss: 0.2144 | g_loss: 3.8747
Epoch [   18/   20] (50%) | d_loss: 0.4029 | g_loss: 3.7017
Epoch [   18/   20] (57%) | d_loss: 0.1639 | g_loss: 3.9554
Epoch [   18/   20] (64%) | d_loss: 0.0691 | g_loss: 3.3320
Epoch [   18/   20] (71%) | d_loss: 0.1753 | g_loss: 3.7015
Epoch [   18/   20] (78%) | d_loss: 3.4414 | g_loss: 0.9860
Epoch [   18/   20] (85%) | d_loss: 0.1621 | g_loss: 4.3854
Epoch [   18/   20] (92%) | d_loss: 0.4272 | g_loss: 4.7343
Epoch [   18/   20] (100%) | d_loss: 0.2194 | g_loss: 4.4626
Epoch [   19/   20] (0%) | d_loss: 0.6452 | g_loss: 2.8491
Epoch [   19/   20] (7%) | d_loss: 0.1413 | g_loss: 3.5936
Epoch [   19/   20] (14%) | d_loss: 0.1905 | g_loss: 4.8527
Epoch [   19/   20] (21%) | d_loss: 0.0839 | g_loss: 4.1368
Epoch [   19/   20] (28%) | d_loss: 0.0758 | g_loss: 4.0217
Epoch [   19/   20] (36%) | d_loss: 0.3840 | g_loss: 5.1562
Epoch [   19/   20] (43%) | d_loss: 0.1086 | g_loss: 3.8248
Epoch [   19/   20] (50%) | d_loss: 0.1236 | g_loss: 3.9028
Epoch [   19/   20] (57%) | d_loss: 0.0812 | g_loss: 3.9567
Epoch [   19/   20] (64%) | d_loss: 0.1199 | g_loss: 6.4323
Epoch [   19/   20] (71%) | d_loss: 0.0440 | g_loss: 4.5381
Epoch [   19/   20] (78%) | d_loss: 0.0361 | g_loss: 4.4332
Epoch [   19/   20] (85%) | d_loss: 0.0654 | g_loss: 3.9759
Epoch [   19/   20] (92%) | d_loss: 0.0363 | g_loss: 4.6166
Epoch [   19/   20] (100%) | d_loss: 0.6942 | g_loss: 2.9463
Epoch [   20/   20] (0%) | d_loss: 0.5292 | g_loss: 2.2928
Epoch [   20/   20] (7%) | d_loss: 0.0799 | g_loss: 4.1265
Epoch [   20/   20] (14%) | d_loss: 0.3047 | g_loss: 2.7846
Epoch [   20/   20] (21%) | d_loss: 0.2332 | g_loss: 3.4602
Epoch [   20/   20] (28%) | d_loss: 0.6324 | g_loss: 2.4507
Epoch [   20/   20] (36%) | d_loss: 0.1390 | g_loss: 3.3646
Epoch [   20/   20] (43%) | d_loss: 0.2344 | g_loss: 6.0528
Epoch [   20/   20] (50%) | d_loss: 0.0685 | g_loss: 5.0674
Epoch [   20/   20] (57%) | d_loss: 0.0693 | g_loss: 3.9079
Epoch [   20/   20] (64%) | d_loss: 0.4699 | g_loss: 1.8186
Epoch [   20/   20] (71%) | d_loss: 0.1149 | g_loss: 3.2995
Epoch [   20/   20] (78%) | d_loss: 0.2089 | g_loss: 2.8690
Epoch [   20/   20] (85%) | d_loss: 0.1691 | g_loss: 4.3219
Epoch [   20/   20] (92%) | d_loss: 0.3789 | g_loss: 3.2550
Epoch [   20/   20] (100%) | d_loss: 0.0766 | g_loss: 4.2285

Training loss

Plot the training losses for the generator and discriminator, recorded after each epoch.

In [21]:
fig, ax = plt.subplots()
losses = np.array(losses)
plt.plot(losses.T[0], label='Discriminator', alpha=0.5)
plt.plot(losses.T[1], label='Generator', alpha=0.5)
plt.title("Training Losses")
plt.legend()
Out[21]:
<matplotlib.legend.Legend at 0x1a22c6c0710>

Generator samples from training

View samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.

In [42]:
# helper function for viewing a list of passed in sample images
def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)        
        im = ax.imshow(img.reshape((64,64,3)))
    fig.suptitle('Epoch'+str(epoch), fontsize=16)
    return fig
In [43]:
# Load samples from generator, taken while training
with open('train_samples_19.pkl', 'rb') as f:
    samples = pkl.load(f)
In [44]:
for i in range(20):
    view_samples(i, samples)